fix: Unable to acquire impersonated credentials by hsuyuming · Pull Request #2003 · google/adk-python

hsuyuming · 2025-07-16T22:36:16Z

add scope "https://www.googleapis.com/auth/cloud-platform" within google.auth.default

hsuyuming · 2025-07-16T22:38:22Z

@hangfei @seanzhou1023
This is my PR in order to fix #2001, please help me review it, if there are any part need to be modified, please let me know. Thank you

hsuyuming · 2025-07-16T22:43:00Z

hangfei · 2025-07-19T19:43:15Z

Thanks for the contribution. Please follow https://github.com/google/adk-python/blob/main/CONTRIBUTING.md.

Allaway11 · 2025-07-22T14:16:29Z

We also are stuck at this point with authenticating with a service account. would be great to get this merged.

hsuyuming · 2025-07-22T22:08:55Z

I will try to modify my PR before the end of this wek.

hsuyuming · 2025-07-25T19:14:10Z

Unit Test successfully result:

============================= test session starts ==============================
platform linux -- Python 3.12.10, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/user/abehsu/adk-python-hsuyuming
configfile: pyproject.toml
plugins: asyncio-1.1.0, mock-3.14.1, anyio-4.9.0, langsmith-0.3.45, xdist-3.8.0
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collected 3786 items

tests/unittests/a2a/converters/test_event_converter.py ................. [  0%]
...........................................................              [  2%]
tests/unittests/a2a/converters/test_part_converter.py .................. [  2%]
..................................................                       [  3%]
tests/unittests/a2a/converters/test_request_converter.py ............... [  4%]
...........                                                              [  4%]
tests/unittests/a2a/converters/test_utils.py ........................... [  5%]
.............                                                            [  5%]
tests/unittests/a2a/executor/test_a2a_agent_executor.py ................ [  5%]
..............................                                           [  6%]
......
tests/unittests/tools/test_from_function_with_options.py ............... [ 90%]
.                                                                        [ 90%]
tests/unittests/tools/test_function_tool.py ............................ [ 91%]
..........                                                               [ 91%]
tests/unittests/tools/test_gemini_schema_util.py ....................... [ 91%]
........................................................................ [ 93%]
.................                                                        [ 94%]
tests/unittests/tools/test_google_search_tool.py ....................... [ 94%]
...............                                                          [ 95%]
tests/unittests/tools/test_langchain_tool.py ........                    [ 95%]
tests/unittests/tools/test_long_running_tool.py ....................     [ 96%]
tests/unittests/tools/test_url_context_tool.py ......................... [ 96%]
.....                                                                    [ 96%]
tests/unittests/tools/test_vertex_ai_search_tool.py .................... [ 97%]
..................                                                       [ 97%]
tests/unittests/utils/test_feature_decorator.py ........................ [ 98%]
..                                                                       [ 98%]
tests/unittests/utils/test_instructions_utils.py ......................  [ 99%]
tests/unittests/utils/test_model_name_utils.py ......................... [ 99%]
.........                                                                [100%]
================ 3786 passed, 2100 warnings in 67.78s (0:01:07) ================

End to End testing result:

Impersonated testing (before fix)

gcloud settings

(adk_with_different_auth) user@abehsu-us-vscode:~/abehsu/adk_with_different_auth$ gcloud config list
[auth]
impersonate_service_account = shared-service-account@<gcp project>.iam.gserviceaccount.com
[core]
account = xxxxx
disable_usage_reporting = True
project = <gcp project>

Your active configuration is: [gcp project]

application_default_credentials.json

(adk-python-hsuyuming) user@abehsu-us-vscode:~/abehsu/adk_with_different_auth/oauth2/apihub_tools$ cat ~/.config/gcloud/application_default_credentials.json 
{
  "delegates": [],
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/shared-service-account@xxxxxx.iam.gserviceaccount.com:generateAccessToken",
  "source_credentials": {
    "account": "",
    "client_id": "xxxxxxx",
    "client_secret": "xxxxx",
    "refresh_token": "xxxxxx",
    "type": "authorized_user",
    "universe_domain": "googleapis.com"
  },
  "type": "impersonated_service_account"
}

testing code

from google.adk.tools.apihub_tool.apihub_toolset import APIHubToolset
from google.adk.agents.llm_agent import LlmAgent

userinfo_toolset = APIHubToolset(
  name="enterprise_assistant",
  description="enterprise assistant tool",
  #  api: abehsu_test
  apihub_resource_name="projects/<gcp project>/locations/us-west1/apis/<resource>", # API Hub resource name
)

# --- Agent Configuration ---
# Configure and create the main LLM Agent.
root_agent = LlmAgent(
    model='gemini-2.0-flash',
    name='enterprise_assistant',
    instruction='Help user integrate with multiple enterprise systems, including retrieving user information which may require authentication.',
    tools=[userinfo_toolset],
)

lunch adk web

adk web

user start communication

error log

(adk-python-hsuyuming) user@abehsu-us-vscode:~/abehsu/adk_with_different_auth/oauth2$ adk web
/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/fast_api.py:365: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super().__init__()
INFO:     Started server process [17416]
INFO:     Waiting for application startup.

+-----------------------------------------------------------------------------+
| ADK Web Server started                                                      |
|                                                                             |
| For local testing, access at http://localhost:8000.                         |
+-----------------------------------------------------------------------------+

INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:60566 - "GET / HTTP/1.1" 307 Temporary Redirect
INFO:     127.0.0.1:60566 - "GET /dev-ui/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:60566 - "GET /dev-ui/main-SRBSE46V.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:60560 - "GET /dev-ui/chunk-EQDQRRRY.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:60576 - "GET /dev-ui/styles-4VDSPQ37.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:60578 - "GET /dev-ui/polyfills-B6TNHZQ6.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:60578 - "GET /dev-ui/assets/config/runtime-config.json HTTP/1.1" 200 OK
INFO:     127.0.0.1:60576 - "GET /dev-ui/assets/ADK-512-color.svg HTTP/1.1" 200 OK
INFO:     127.0.0.1:60578 - "GET /list-apps?relative_path=./ HTTP/1.1" 200 OK
2025-07-25 18:49:16,256 - INFO - fast_api.py:486 - New session created
INFO:     127.0.0.1:60578 - "POST /apps/apihub_tools/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:60566 - "GET /apps/apihub_tools/eval_results HTTP/1.1" 200 OK
INFO:     127.0.0.1:60576 - "GET /apps/apihub_tools/eval_sets HTTP/1.1" 200 OK
INFO:     127.0.0.1:60578 - "GET /apps/apihub_tools/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:60578 - "GET /apps/apihub_tools/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:60578 - "GET /.well-known/appspecific/com.chrome.devtools.json HTTP/1.1" 404 Not Found
INFO:     127.0.0.1:36404 - "POST /run_sse HTTP/1.1" 200 OK
2025-07-25 18:49:27,687 - INFO - envs.py:47 - Loaded .env file for apihub_tools at /home/user/abehsu/adk_with_different_auth/oauth2/apihub_tools/.env
2025-07-25 18:49:27,688 - INFO - envs.py:47 - Loaded .env file for apihub_tools at /home/user/abehsu/adk_with_different_auth/oauth2/apihub_tools/.env
2025-07-25 18:49:28,951 - ERROR - fast_api.py:889 - Error in event_generator: ("Fail to load 'apihub_tools' module. Unable to acquire impersonated credentials", '{\n  "error": {\n    "code": 400,\n    "message": "Request contains an invalid argument.",\n    "status": "INVALID_ARGUMENT"\n  }\n}\n')
Traceback (most recent call last):
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/fast_api.py", line 876, in event_generator
    runner = await _get_runner_async(req.app_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/fast_api.py", line 1027, in _get_runner_async
    root_agent = agent_loader.load_agent(app_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/utils/agent_loader.py", line 198, in load_agent
    agent = self._perform_load(agent_name)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/utils/agent_loader.py", line 172, in _perform_load
    if root_agent := self._load_from_module_or_package(agent_name):
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/utils/agent_loader.py", line 92, in _load_from_module_or_package
    raise e
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/cli/utils/agent_loader.py", line 60, in _load_from_module_or_package
    module_candidate = importlib.import_module(agent_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.12.10-linux-x86_64-gnu/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/user/abehsu/adk_with_different_auth/oauth2/apihub_tools/__init__.py", line 1, in <module>
    from . import agent
  File "/home/user/abehsu/adk_with_different_auth/oauth2/apihub_tools/agent.py", line 6, in <module>
    userinfo_toolset = APIHubToolset(
                       ^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/tools/apihub_tool/apihub_toolset.py", line 149, in __init__
    self._prepare_toolset()
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/tools/apihub_tool/apihub_toolset.py", line 169, in _prepare_toolset
    spec_str = self._apihub_client.get_spec_content(self._apihub_resource_name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/tools/apihub_tool/clients/apihub_client.py", line 95, in get_spec_content
    api = self.get_api(apihub_resource_name)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/tools/apihub_tool/clients/apihub_client.py", line 151, in get_api
    "Authorization": f"Bearer {self._get_access_token()}",
                               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/adk/tools/apihub_tool/clients/apihub_client.py", line 339, in _get_access_token
    credentials.refresh(Request())
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/auth/impersonated_credentials.py", line 260, in refresh
    self._update_token(request)
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/auth/impersonated_credentials.py", line 325, in _update_token
    self.token, self.expiry = _make_iam_token_request(
                              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/abehsu/adk_with_different_auth/.venv/lib/python3.12/site-packages/google/auth/impersonated_credentials.py", line 100, in _make_iam_token_request
    raise exceptions.RefreshError(_REFRESH_ERROR, response_body)
google.auth.exceptions.RefreshError: ("Fail to load 'apihub_tools' module. Unable to acquire impersonated credentials", '{\n  "error": {\n    "code": 400,\n    "message": "Request contains an invalid argument.",\n    "status": "INVALID_ARGUMENT"\n  }\n}\n')
INFO:     127.0.0.1:36404 - "GET /apps/apihub_tools/users/user/sessions/c06b7454-696f-4ab5-890f-8d535f6701b0 HTTP/1.1" 200 OK
INFO:     127.0.0.1:36414 - "GET /debug/trace/session/c06b7454-696f-4ab5-890f-8d535f6701b0 HTTP/1.1" 200 OK

Impersonated testing (after fix)

gcloud settings

(adk_with_different_auth) user@abehsu-us-vscode:~/abehsu/adk_with_different_auth$ gcloud config list
[auth]
impersonate_service_account = shared-service-account@<gcp project>.iam.gserviceaccount.com
[core]
account = xxxxx
disable_usage_reporting = True
project = <gcp project>

Your active configuration is: [gcp project]

lunch adk web

adk web

no error appear

normal account testing (after fix)

gcloud config settings

(adk-python-hsuyuming) user@abehsu-us-vscode:~/abehsu/adk_with_different_auth/oauth2/apihub_tools$ gcloud config list
[core]
account = <account>
disable_usage_reporting = True
project = <gcp project>

Your active configuration is: [gdw-dev-apigee-user]

application_default_credentials.json

(adk-python-hsuyuming) user@abehsu-us-vscode:~/abehsu/adk_with_different_auth/oauth2/apihub_tools$ cat ~/.config/gcloud/application_default_credentials.json 
{
  "account": "",
  "client_id": "xxxxx",
  "client_secret": "xxxx",
  "quota_project_id": "xxxxx",
  "refresh_token": "xxxxx",
  "type": "authorized_user",
  "universe_domain": "googleapis.com"
}

lunch adk web

adk web

successfully result

hsuyuming · 2025-07-25T19:15:36Z

Hi @hangfei :
I update my PR to reach all of the requirements within CONTRIBUTING.md, can you help me review once?

Merge #2003 add scope "https://www.googleapis.com/auth/cloud-platform" within google.auth.default COPYBARA_INTEGRATE_REVIEW=#2003 from hsuyuming:fix/issue_2001_support_impersonated_credential 8874a36 PiperOrigin-RevId: 788656025

hangfei · 2025-07-29T23:52:19Z

@hsuyuming this is merged by bot. thanks!

hsuyuming · 2025-07-30T04:17:23Z

Thank you @hangfei, I have also closed the original issue.
I appreciate the support from you and your team.

Merge google#2003 add scope "https://www.googleapis.com/auth/cloud-platform" within google.auth.default COPYBARA_INTEGRATE_REVIEW=google#2003 from hsuyuming:fix/issue_2001_support_impersonated_credential 8874a36 PiperOrigin-RevId: 788656025

hsuyuming force-pushed the fix/issue_2001_support_impersonated_credential branch 3 times, most recently from 1efbdae to 10e39b1 Compare July 25, 2025 18:28

Fix: Support impersonated credential (google#2001)

61c42db

hsuyuming force-pushed the fix/issue_2001_support_impersonated_credential branch from 10e39b1 to 61c42db Compare July 25, 2025 18:36

Merge branch 'main' into fix/issue_2001_support_impersonated_credential

8874a36

seanzhou1023 approved these changes Jul 29, 2025

View reviewed changes

seanzhou1023 added the ready to pull [Status] This PR is ready to be imported back to Google label Jul 29, 2025

hangfei closed this Jul 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Unable to acquire impersonated credentials#2003

fix: Unable to acquire impersonated credentials#2003
hsuyuming wants to merge 2 commits into
google:mainfrom
hsuyuming:fix/issue_2001_support_impersonated_credential

hsuyuming commented Jul 16, 2025

Uh oh!

hsuyuming commented Jul 16, 2025

Uh oh!

hsuyuming commented Jul 16, 2025 •

edited

Loading

Uh oh!

hangfei commented Jul 19, 2025

Uh oh!

Allaway11 commented Jul 22, 2025

Uh oh!

hsuyuming commented Jul 22, 2025

Uh oh!

hsuyuming commented Jul 25, 2025 •

edited

Loading

Uh oh!

hsuyuming commented Jul 25, 2025

Uh oh!

hangfei commented Jul 29, 2025

Uh oh!

hsuyuming commented Jul 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

hsuyuming commented Jul 16, 2025

Uh oh!

hsuyuming commented Jul 16, 2025

Uh oh!

hsuyuming commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hangfei commented Jul 19, 2025

Uh oh!

Allaway11 commented Jul 22, 2025

Uh oh!

hsuyuming commented Jul 22, 2025

Uh oh!

hsuyuming commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test successfully result:

End to End testing result:

Impersonated testing (before fix)

Impersonated testing (after fix)

normal account testing (after fix)

Uh oh!

hsuyuming commented Jul 25, 2025

Uh oh!

hangfei commented Jul 29, 2025

Uh oh!

hsuyuming commented Jul 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hsuyuming commented Jul 16, 2025 •

edited

Loading

hsuyuming commented Jul 25, 2025 •

edited

Loading